Fast kernel matrix-vector multiplication with application to Gaussian process learning

A number of core computational problems in machine learning, both old and new, can be cast as a matrixvector multiplication between a kernel matrix or class-probability matrix and a vector of weights. This arises prominently, for example, in the kernel estimation methods of nonparametric statistics, many common probabilistic graphical models, and the more recent kernel machines. After highlighting the existence of this computational problem in several well-known machine learning methods, we focus on a solution for one specific example for clarity, Gaussian process (GP) prediction one whose applicability has been particularly hindered by this computational barrier. We demonstrate the application of a recent JV-body approach developed specifically for statistical problems, employing adaptive computational geometry and finite-difference approximation. This core algorithm reduces the O(N) matrix-vector multiplications within GP learning to O(N), making the resulting overall learning algorithm O(N). GP learning for N = 1 million points is demonstrated. 1 Kernel Matrix-Vector Multiplications in Learning A kernel matrix $ contains the kernel interaction of each point x^ in a query (test) dataset 2LQ (having size NQ) with each point from a reference (training) dataset X_n (having size iV^), where the kernel function K() often has some scale parameter a (the 'bandwidth'). Often the 'kernel function' is actually a probability density function, such as the Gaussian. In such cases $ is typically a class probability matrix. The query and reference set can be the same set. Often the core computational cost of a statistical method boils down to a multiplcation of this matrix $ with some vector of weights w. For example, in the weighted form of kernel density estimation, the density estimate at the q test point x_q is